Papers with gradient-level optimization
Por Qué Não Utiliser Alla Språk? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | a lack of labeled data for low-resource languages leads to the need for effective cross-lingual transfer learning. |
| Approach: | They propose a mixed training method that trains on both source and target data with stochastic gradient surgery, a novel gradient-level optimization. |
| Outcome: | The proposed method outperforms current methods on all tasks and escapes overfitting issues. |